Rényi Entropy , Guesswork Moments and Large Deviations C - E Pfister and WG Sullivan

For a large class of stationary probability measures on AN, where A is a finite alphabet, we compute the specific Rényi entropy of order α and the specific guesswork moments of order β > −1. We show that the specific guesswork moment of order β equals the specific Rényi entropy of order α = 1/(1+ β) multiplied by β. The method is based on energy–entropy estimates suggested by statistical physics. The technique also yields a simple proof of the large deviation principle for the empirical measure on the space of an irreducible sofic shift with reference probability measure ν, which is stationary and satisfies a rate condition on the probability of allowed words. Keywords— Guesswork, Renyi entropy, large deviation, sofic shift.